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FITTING  FUNCTIONAL  EQUATIONS  TO  EXPERIMENTAL  DATA 

s|c 

H.  Kagiwada  and  R.  Kalaba 
The  RAND  Corporation,  Santa  Monica,  California 

1.  Introduction 

Much  of  mathematical  analysis  is  devoted  to  the 
problem  of  predicting  the  future  behavior  of  a  system, 
given  a  descriptive  equation  and  the  current  state.  This 
is  surprising  since  a  basic  scientific  problem  in  such 
fields  as  physics,  engineering,  biology  and  economics  is 
that  of  determining  the  structure  of  a  system,  given 
various  observations  over  time  [l-6].  In  the  past,  such 
system  identification  or  inverse  problems  were  solved  by 
trial  and  error  or  by  simple  approximations.  Modern 
computing  machines  have  dramatically  altered  the  picture 
with  their  ability  to  integrate  systems  of  several 
thousand  differential  equations  given  a  complete  set  of 
initial  conditions.  Many  types  of  functional  equations 
may  be  converted  into  systems  of  ordinary  differential 
equations.  This  means  that  wide  classes  of  direct 
problems  can  be  solved  as  initial-value  problems. 

$ 
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ThlB  also  means  that  a  great  many  inverse  problems 
may  be  computationally  resolved.  Let  the  equations  which 
describe  a  particular  pr >cess  be  a  system  of  differential 
equations.  The  unknown  structure  of  the  process  is 
reflected  in  unknown  system  parameters  which  appear  in 
tho  differential  equations  or  in  the  initial  conditions. 
These  parameters  are  to  be  estimated  on  the  basis  of 
observations  of  the  process.  The  identification  problem 
takes  the  form  of  a  nonlinear  boundary- value  problem. 

Tins  can  be  solved  by  a  variety  of  methods.  One  that 
has  been  shown  to  be  quite  effective  is  described  below. 

2.  Illustrative  Example 

Consider  a  system  undergoing  a  process  which  may  be 
described  by  the  scalar  differential  equation 

(1)  *  =  g(x,  a), 

where  a  is  an  unknown  system  parameter,  and  the  initial 
state 

(2)  x (0)  »  c 

is  also  unknown.  Suppose  that  at  the  time  tj  the  system 
is  observed  to  be  in  the  state  b^,  i  ■  1,  2,  N;  t^+1 

Neither  the  observations  nor  the  theory  of  Eq.  (1)  need 
be  perfect,  so  in  general 


(3) 


x(tl)  *  bt, 


i  ■  1 ,  2 , 


N. 
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To  fit  the  theoretical  equation  (1)  to  the  experimental 

data,  we  wish  to  determine  the  constants  a  and  c  so  that 

we  minimize  S,  the  sum  of  the  squares  of  the  deviations 

N 

(4)  SO  (x(tt)  -  b±]2 . 

i=l 

Frequently  this  can  be  done  using  a  quadrat ical ly 
convergent  successive  approximation  scheme.  Let  x^Ct), 
aQ  and  cQ  be  the  current  approximations  to  the  optimal 
function  x(t)  and  the  parameters  a  and  c.  The  new 
approximations  are  obtained  by  consi  aring  the  linearized 
equation 

(5)  xr  *  g(x0,  aQ)  +  (x1  -  x0)gx(x0,  »0) 

4*  (iij  *  a(j)gft  (xq,  a^)  • 

The  solution  of  this  equation  may  be  written  in  the  form 

(6)  x j (t)  =  p(t)  +  Cjhft)  +  a^t),  0  <  t  <  tN, 

where  the  functions  p,  h,  and  q  are  solutions  of  the  linear 
initial-value  problems 

(7)  p  =  *x(x0»  a0)P  +  *(xo»  ao^  ~  x0  £x^x0’  a0^ 

-  ao*a(x0»  “O*'  p<0)  *  °» 

(8)  h  a  BxC*o»  ao^b»  h<0)  a 

(9)  <i  -  gx(x0,  aQ)q  +  ga(xQ,  a0),  q(0)  -0,  tj  s  t  n  tN. 


The  functions  p,  h  and  q  are  readily  determined  numerically 


Then  the  constants  c1  and  a^  are  determined  by  minimizing 
the  expression 

N  2 
(10)  E  ■  }  “(p C t +  c jh(t^)  +  o1q(ti)  -  bi|  , 

i«l 

which  involves  solving  the  linear  algebraic  equations 


(U) 


«?  0 . 


The  process  is  repeated  until  a  sufficiently  small  change 
takes  place  from  one  step  to  the  next.  In  this  way, 
frequently,  the  differential  equation  may  be  fitted  to 
the  experimental  data  [4]. 


3.  Other  Functional  Equations 

Systems  of  ordinary  differential  equations  may  be 
fitted  to  the  data  in  the  manner  /fust  described.  So  may 
functional  equations  which  can  be  reduced  exactly  or 
approximately  to  systems  of  ordinary  differential  equations 
The  wave  equation,  for  example, 

(12)  utt  »  c2(x)uxx 

can  be  reduced  to  ordinary  differential  equations  either 
by  taking  Laplace  transforms  or  by  using  the  semi- 
discretization  technique.  This  leads  to  the  possibility 
of  identifying  inhomogeneous  media  through  observation 
of  wave  propagation  processes  [7].  The  reduction  of 
differential-difference  equations  to  ordinary  differential 


equations  and  the  estimation  of  time  lags  are  discussed 
in  (5).  Even  integral  equations  may  be  converted  into 
initial  value  problems  { 8 ) ;  this  leads  to  the  computational 
solution  of  many  inverse  problems  in  multiple  scattering 

[6j. 


4.  Discussion 

The  leant  squares  criterion  of  the  above  example  may 
be  replaced  by  some  other  criterion,  such  as  a  minimax 
condition  [9],  depending  on  the  statistical  nature  of 
the  problem  and  any  a  priori  estimates  that  are  available. 

If  the  nonlinear  Eq .  (1)  represents  a  large  system 
of  differential  equations  with  complicated  right-hand 
sides,  the  forming  of  the  partial  derivatives  needed  in 
the  linearized  equations  could  be  a  formidable  task.  A 
method  for  the  computer  evaluation  of  partial  derivatives 
of  functions  given  analytically  has  been  described  by 
Wengert  [10]  and  shown  to  be  useful  in  orbit  determi¬ 
nation  [  11 1  . 

The  linearized  differential  equations  may  be  unstable. 
The  method  of  invariant  imbedding  provides  a  reformulation 
in  terms  of  an  initial-value  problem  for  stable  differ¬ 
ential  equations  [l2j.  Invariant  labeling  also  obviates 
the  need  to  solve  linear  algebraic  equations,  another 
source  of  difficulty,  by  enabling  the  unknown  constants 
to  be  directly  determined. 
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If  the  data  are  continuously  received,  an  invariant 
imbedding  method  for  sequential  filtering  may  be  employed 
(13,  14] . 


-7. 


REFERENCES 


1.  Bellman,  R. ,  H.  Kagiwada,  and  R.  Xalaba,  "Inverse 

Problems  in  Ecology/’  J.  Theor ■  Biology,  Vol .  11, 
part  1,  1966,  pp.  164- t^T1. 

2.  Bellman,  R.,  C.  Collier,  H.  Kagiwada,  R.  Kalaba,  and 

R.  Selves ter,  "Estimation  of  Heart  Parameters  Using 
Skin  Potential  Measurements/'  Comm.  ACM,  Vol.  7, 

1964,  pp.  666-668 . 

3.  Bellman,  R.,  J.  Jacquez,  R,  Kalaba,  and  S.  Schwimmer, 

"Quasilinearization  and  the  Estimation  of  Chemical 
Rate  Constants  from  Raw  Kinetic  Data,"  Math.  Bio- 
sciences,  Vol.  1,  1967,  pp.  71-76.  *” 

4.  Bellman,  R.,  ind  R.  Kalaba,  Quasil Inear izatlon  and 

Nonlinear  Boundary-value  Problems ,  American  felsevler 
PuET  ishing  Co .  ,  New  York,  IWIT 

5.  Bellman,  R.,  H.  Kagiwada,  and  R.  Kalaba,  "Quasilinear¬ 

ization  and  the  Estimation  of  Time  Lags,"  Math.  Blo- 
sciences,  Vol.  1,  1967,  pp.  39-44  ” 

6.  Bellman,  R.,  H.  Kagiwada,  R.  Kalaba,  and  S.  Ueno, 

"Inverse  Problems  in  Radiative  Transfer:  Layered 
Media,"  Icarus ,  Vol.  4,  1965,  pp.  119-126. 

7.  Bellman,  R,,  D .  De tchmendy ,  H,  Kagiwada,  and  R.  KaJ aba, 

On  the  Identification  of  Systems  and  the  Unscrambling 
of  Data--lII:  One-dimensional  Nave  and  Diffusion 
Processes.  The  RAND  Corporation,  Rif«5173-PB. 

October  1966. 

8.  Kagiwada,  H.,  and  R.  Kalaba,  An  Initial  Value  Method 

for  Fredholm  Integral  Equations  of  Convolution  Type , 
■fhe  RAn£>  Corporation,  KM- 5 186- PR,  Novembe r  1 3 . 

9.  Bellman,  R.,  H.  Kagiwada,  and  R.  Kalaba,  "Quasilinear¬ 

ization,  Boundary-value  Problems  and  Linear  Program¬ 
ming/'  IEEE  Trans.  Automatic  Control,  Vol.  AC-10, 

1965,  pT'TSS" 

10.  Wengert,  R.,  "A  Simple  Automatic  Derivative  Evaluation 

Program,"  Comm.  ACM.  Vol.  7,  1964,  pp.  463-464. 

11.  Bellman,  R.,  H.  Kagiwada,  and  R.  Kalaba,  "Wengert's 

Numerical  Method  for  Partial  Derivatives,  Orbit 
Determination  and  Quasilinearization,"  Comm.  ACM, 

Vol.  8,  1965,  pp.  231-232. 


8- 


12.  Bellman,  R . ,  H.  Kagiwada,  and  R.  Kalaba,  "Invariant 

Imbedding  and  the  fhnnerienl  Integration  of  Boundary- 
value  Problems  for  Unstable  Linear  Systems  of 
Ordinary  Differential  Equations,”  Comm.  ACM,  Vol . 

10,  1967,  pp.  100-102, 

13.  Bellman,  R.,  H.  Kagiwada,  R.  Kalaba,  and  R.  Sridhar, 

"Invariant  Imbedding  and  Nonlinear  Filtering  Theory," 

J.  Astronaut,  Sci.,  Vol.  13,  1966,  pp.  110-115, 

14.  Kagiwada,  H.,  R,  Kalaba,  A.  Sehumitzky,  and  R.  Sridhar, 

"Invariant  Imbedding  and  Sequential  Interpolating 
Filters  for  Nonlinear  Processes,"  The  RAND  Corporation, 
HU- 5507- PR,  October  1967. 


